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THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 . 1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 
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Status 

1 Responsive to communication(s) filed on 26 July 2004 . 
2a)KI This action is FINAL. 2b)n This action is non-final. 

3) 0 Since this application is in condition for allowance except for formal nnatters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 
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4) |EI Claim(s) 24-40 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) 0 Claim(s) is/are allowed. 

6) IE1 Claim(s) 24-40 is/are rejected. 

Claim(s) is/are objected to. 

8) n Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) 0 The specification is objected to by the Examiner. 

10)0 The drawing(s) filed on is/are: a)[J accepted or b)n objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held In abeyance. See 37 CFR 1.85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
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application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 
Arguments and Amendments 

1 . Applicants' amendments and arguments filed 26 July 2004, have been entered and made 
of record. 

Response to Amendments and Arguments 

2. Applicant has canceled claims 1-23 and introduced new claims 24-40. Treatment of new 
claims 24-40 is found below. 

3. In regards to previous claims now corresponding to new claims 27, 28, 36, and 37, 
Applicant argues (page 6 of Response) that Waxman does not disclose a "simple color fusion 
(SCF)" algorithm or a "principal component color fusion (PCCF)" algorithm, as claimed. 
Applicant has only nominally claimed utilizing SCF and PCCF and has not claimed any specific 

V 

features or limitations regarding these two algorithms. Given the broadest reasonable 
interpretation of these two algorithms, one need only to show a "simple" color fusion algorithm 
or a fusion algorithm that involves "principle color components." 

Nevertheless, the Examiner has referred to the Specification to try to ascertain, as closely 
as possible, what these two algorithms encompass. According to the Specification, SCF "takes 
outputs . . . and assigns these to the colors . . . based on their respective wavelengths" (page 7). 
Similarly, Waxman in figure 6 illustrates an algorithm that assigns colors to outputs based on the 
outputs' respective wavelengths, such as visible/near IR wavelengths, and long-wave IR 
wavelengths. Therefore, a "simple color fusion algorithm" reads on figure 6 of Waxman. 

The Specification denotes PCCF as rotating each pixel value "into a coordinate frame in 
which the principle component . . . aligns with the brightness" (page 7), Similarly, Waxman 
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discloses converting RGB color codes into the HVS color space, wherein the principal color 
component H is aUgned with the brightness. See colxmm 3, lines 28-43. Therefore, Waxman 
discloses a "principal color component fusion algorithm." 

4. In regards to previous claims now corresponding to new claims 29-31 and 38, Applicant 
argues (page 7 of Response) that there is no motivation to combine Waxman with Watkins to 
employ a third sensor, as claimed. Watkins teaches that, when fusing together captured images 
from a plurality of image sensors, it is preferable to employ three sensor pairs, each of which is 
sensitive a different spectral range (column 2, lines 35-43). One skilled in the art would have 
been motivated to modify Waxman by Watkins to utilize three sensors for image fusion, rather 
than only two, because Watkins teaches that utilizing three sensors for the purposes of image 
fusion is conventional, and by using three sensors as opposed to two, image data in a larger 
number of spectral ranges can be acquired. For instance, figure 6 of Waxman shows acquisition 
of visible/near TR image data by one sensor and long-wave IR by another sensor. According to 
Watkins, it is advantageous to include a third sensor that is sensitive e.g. in the mid-range IR 
(column 2, lines 41-43), so that a wider spectral response is obtained. 

5. Applicant also traversed the prior official notice of previous claim 8, which now 
corresponds to claim 29. U.S. Patent 4,533,938 by Hurst has been apphed to provide evidence 
that desaturating an image signal, as claimed, was an obvious expedient. 

Claim Rejections - 35 USC § 102 

6. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 
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A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

7. Claims 24-28, 33-37, and 40 are rejected under 35 U.S.C. 102(b) as being anticipated by 
U.S. Patent 5,555,324 by Waxman et al. ("Waxman"). 

Regarding claims 24 and 33, Waxman discloses an image processing apparatus/method 
(figure 6) for processing imaging data in a plurality of spectral bands and fusing the data into a 
color image, comprising: 

two or more image sensors (310 and 312); 

at least two image-acquiring sensor areas located on said one or more imaging sensors 
(i.e. a first sensor area on camera 310 and a second sensor area on camera 312), wherein each 
said sensor is sensitive to a different spectral band than at least one other of said sensor areas 
(sensor for camera 310 is sensitive to visible-near IR spectral band, and sensor for camera 312 is 
sensitive to long-wave IR spectral band) and generates an image output representative of an 
acquired image in the spectral band to which the sensor area is sensitive (sensors for cameras 310 
and 312 produce visible/near IR and long-wave IR images, respectively); 

a frame grabber connected to said imaging sensors (figure 4: A/D converters 328 digitize 
and transmit frames of image data to the processors); 

a general purpose computer (figure 4: processor 330 is a general-purpose computer that 
performs various functions) connected to said imaging sensors for executing in real time: 

a registration algorithm for scaling and registering said image outputs (vision processor 
392 of figure 4 contains processors 360 and 362, which execute algorithms for scaling, 
registering and generally preprocessing the image outputs); and ' 
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a color fusion algorithm for combining said image outputs into a single image (vision 
processor 392 of figure 4 contains processors 130, 132, and 134, which perform center-surround 

shunt algorithms to combine the two images to create a single RGB image). 

Regarding claim 25 and 34, Waxman discloses a screen display to display the outputs 
(332, figure 4). 

Regarding claims 26 and 35, Waxman discloses an operator interface for allowing 
operator input in processing of said image outputs (column 15, lines 33-37: user input is allowed 
for selecting a color mapping). 

Regarding claims 27 and 36, Waxman discloses the color fusion algorithm is simple color 
fusion (figure 6: Waxman uses RGB color codes based on the wavelengths). 

Regarding claims 28 and 37, Waxman discloses the color fusion algorithm is based on 
principle component color fusion (column 3, lines 28-43: RGB color codes are converted to the 
HVS color space). 

Regarding claim 40, Waxman discloses the processing and fusing of said image occurs in 
real time (column 11, lines 1-4). 

Claim Rejections - 35 USC § 103 
8. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 
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9. This application currently names joint inventors. In considering patentability of the 
claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various 
claims was commonly owned at the time any inventions covered therein were made absent any 
evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out 
the inventor and invention dates of each claim that was not commonly owned at the time a later 
invention was made in order for the examiner to consider the applicability of 35 U.S.C. 103(c) 
and potential 35 U.S.C. 102(e), (f) or (g) prior art under 35 U.S.C. 103(a). 

10. Claim 29 is rejected under 35 U.S.C. 103(a) as being unpatentable over Waxman in view 
of U.S. Patent 4,533,938 by Hurst. 

Regarding claim 29, Waxman discloses performing a PCCF algorithm (i.e. converting 
RGB to HVS values) in order to perform "desirable color manipulations" within the HVS space 
(column 15, lines 5-21). Processor 520, figure 10, is capable of performing color transformations 
to produce desirable effects on the image to be displayed. Waxman does not disclose 
desaturating the HVS image, since Waxman' s shunting algorithms produce images that are 
typically unsaturated (column 8, lines 48-51). However, should the final color image be 
saturated, it would have obvious to utilize processor 520 to desaturate the HVS image using any 
of the well-known desaturation techniques in order to enhance the image for display. 

For example. Hurst discloses an image processing system wherein a desaturation signal is 
added to the image signal when a hue value is modified (see Abstract). This desaturation signal 
is added in order to minimize the changes in saturation when the hue is modified. It would have 
been obvious to one of ordinary skill in the art at the time of the invention to modify Waxman by 
Hurst to desaturate the fused output image, as claimed, since Waxman teaches that a user to 
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allowed to adjust the hue of the output image, and Hurst teaches that, as part of the logistics of 
adjusting the hue of an image signal, it is advantageous to desaturate the image so that changes in 
saturation attributable to the changing of the hue are minimized (see Abstract and column 1, lines 
1 1-28 and column 2, lines 30-38). 

1 1 . Claims 30-32, 38, and 39 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Waxman in view of U.S. Patent 6,597,807 by Watkins et al. ("Watkins"). 

Regarding claim 30, Waxman discloses using a pair of sensors in cameras 310 and 312. 
However, Waxman is silent to including an additional (third) sensor. 

Watkins discloses a similar system for fusing image data received from multiple sensor 
that are sensitive to different spectral ranges. In particular, Watkins discloses an embodiment that 
utilizes three sets of stereo sensor pairs, wherein each pair is sensitive to different spectral ranges 
(column 2, lines 32-43). Then, the images are fused together to form a single RGB image. 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Waxman by Watkins to achieve the claimed invention since utilizing three sensors, as 
taught by Watkins, provides more information of an imaged scene than utilizing only two 
sensors. 

Regarding claims 31 and 38, Waxman discloses using a pair of sensors in cameras 310 
and 312, wherein each sensor maps its image to an associated color channel, and then the 
channels are combined into a color image. However, Waxman is silent to the plurality of sensors 
comprising three sensors. 
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Watkins discloses a similar system for fusing image data received from multiple sensor 
that are sensitive to different spectral ranges. In particular, Watkins discloses an embodiment that 
utilizes three sets of stereo sensor pairs, wherein each pair is sensitive to different spectral ranges 
(column 2, lines 32-43). Then, the images are fused together to form a single RGB image. 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Waxman by Watkins to achieve the claimed invention since utilizing three sensors, as 
taught by Watkins, provides more information of an imaged scene than utilizing only two 
sensors. 

Regarding claims 32 and 39, Waxman discloses the first sensor is sensitive to the visible 
spectral band (310, figure 6), the second sensor is sensitive to the long-wave IR spectral band 
(312, figure 6), and Watkins discloses a third sensor is sensitive to the short-wave spectral band 
(column 2, lines 41-43). 

Conclusion 

12. Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 



Application/Control Number: 09/840,235 



Page 9 



Art Unit: 2623 

CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Colin M. LaRose whose telephone number is (703) 306-3489. 
The examiner can normally be reached Monday through Thursday from 8:00 to 5:30. The 
examiner can also be reached on alternate Fridays. 

If attempts to reach the examiner by telephone are unsuccessftil, the examiner's 
supervisor, Amelia Au, can be reached on (703) 308-6604. The fax phone number for the 
organization where this application or proceeding is assigned is (703) 872-9306. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the TC 2600 Customer Service Office whose telephone number is (703) 



306-0377. 
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